Skip to content

Conversation

@fluidnumerics-joe
Copy link
Contributor

@fluidnumerics-joe fluidnumerics-joe commented Jan 18, 2025

Expected Differences

  • Do these changes introduce new tools, command line arguments, or configuration file options? [Yes or No]

    If yes, please describe:

Yes. This PR brings in a new make file (SRC/FEMAIN/Makefile.hip) that can be used to build FastEddy for AMD GPU accelerated platforms. I've also included a small bit of documentation for building FastEddy on AMD GPU systems, using this new make file. These changes do not require any modification to source code in main_v2.0, as the instructions provided in Makefile.hip leverage ROCm's hipify-clang to convert CUDA code to HIP code prior to compilation with amdclang++/hipcc.

Pull Request Testing

  • Describe testing already performed for these changes:

I've run the convective boundary layer example on both Nvidia V100 and AMD MI210 systems. @jsauer-NCAR has reviewed model output from both runs and confirmed solutions are in agreement.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    Reviewers are invited to obtain an account on our system to test FastEddy on AMD MI210 GPUs, should they want to run existing tutorials on the AMD platform. Please reach out to me directly to get an account.

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes or No]

Yes. However, I could imagine it being worthwhile to include docs on how to run on systems with AMD GPUs (in addition to the build instructions). To this end, we can provide run instructions for our cluster (Galapagos). I might be able to request cycles on Setonix to put together run instructions on Pawsey Supercomputing Centre's Setonix. Down the road, if users or maintainers wish to run on Frontier, we'd be happy to join in on an allocation request for Frontier and help develop build and run documentation specifically for Frontier.

  • Do these changes include sufficient testing updates? [Yes or No]
    No. This is only due to the lack of availability of CI platforms that offer AMD GPUs for testing. Fluid Numerics' Galapagos cluster could do this, but we would need to discuss a support contract to provide GPU-hours dedicated for CI connected via Github Actions and our superci tool.

  • Will this PR result in changes to the test suite? [Yes or No]

    If yes, describe the new output and/or changes to the existing output:

Not presently.

  • Please complete this pull request review by January 28, 2025.

Pull Request Checklist

  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Select: Reviewer(s)
  • After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@fluidnumerics-joe
Copy link
Contributor Author

Hey @jsauer-NCAR , I suspect y'all are busy, but wanted to reach out to let you know this PR is ready. It just adds the Makefile.hip and some basic build documentation for AMD GPUs

@fluidnumerics-joe fluidnumerics-joe changed the base branch from main_v2.0 to develop January 27, 2025 16:29
fluidnumerics-joe and others added 2 commits January 27, 2025 09:29
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
@jsauer-NCAR jsauer-NCAR self-requested a review January 27, 2025 20:33
Copy link
Collaborator

@jsauer-NCAR jsauer-NCAR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this novel contribution Joe.

@jsauer-NCAR jsauer-NCAR merged commit 0ab006f into NCAR:develop Jan 27, 2025
1 check passed
@jprestop jprestop mentioned this pull request Feb 19, 2025
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants